deltaBLEU: A Discriminative Metric for Generation Tasks with Intrinsically Diverse Targets
نویسندگان
چکیده
We introduce Discriminative BLEU (∆BLEU), a novel metric for intrinsic evaluation of generated text in tasks that admit a diverse range of possible outputs. Reference strings are scored for quality by human raters on a scale of [−1, +1] to weight multi-reference BLEU. In tasks involving generation of conversational responses, ∆BLEU correlates reasonably with human judgments and outperforms sentence-level and IBM BLEU in terms of both Spearman’s ρ and Kendall’s τ .
منابع مشابه
∆BLEU: A Discriminative Metric for Generation Tasks with Intrinsically Diverse Targets
We introduce Discriminative BLEU (∆BLEU), a novel metric for intrinsic evaluation of generated text in tasks that admit a diverse range of possible outputs. Reference strings are scored for quality by human raters on a scale of [−1, +1] to weight multi-reference BLEU. In tasks involving generation of conversational responses, ∆BLEU correlates reasonably with human judgments and outperforms sent...
متن کاملA sequence of targets toward a common best practice frontier in DEA
Original data envelopment analysis models treat decision-making units as independent entities. This feature of data envelopment analysis results in significant diversity in input and output weights, which is irrelevant and problematic from the managerial point of view. In this regard, several methodologies have been developed to measure the efficiency scores based on common weights. Specificall...
متن کاملChallenges to Design and Develop of DNA Aptamers for Protein Targets. I. Optimization of Asymmetric PCR for Generation of a Single Stranded DNA Library
Aptamers, or single stranded oligonucleotides, are produced by systematic evolution of ligands by exponential enrichment, abbreviated as SELEX. In the amplification and regeneration step of SELEX technique, dsDNA is conversed to ssDNA. Asymmetric PCR is one of the methods used for the generation of ssDNA. The purpose of this study was to design a random DNA library for selection of aptamers wit...
متن کاملCNN Fixations: An unraveling approach to visualize the discriminative image regions
Deep convolutional neural networks (CNN) have revolutionized various fields of vision research and have seen unprecedented adoption for multiple tasks such as classification, detection, captioning, etc. However, they offer little transparency into their inner workings and are often treated as black boxes that deliver excellent performance. In this work, we aim at alleviating this opaqueness of ...
متن کاملDiscriminative Tracking by Metric Learning
We present a discriminative model that casts appearance modeling and visual matching into a single objective for visual tracking. Most previous discriminative models for visual tracking are formulated as supervised learning of binary classifiers. The continuous output of the classification function is then utilized as the cost function for visual tracking. This may be less desirable since the f...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2015